<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>nm-cloud-setup: NetworkManager Reference Manual</title>
<meta name="generator" content="DocBook XSL Stylesheets Vsnapshot">
<link rel="home" href="index.html" title="NetworkManager Reference Manual">
<link rel="up" href="manpages.html" title="Part I. Manual Pages">
<link rel="prev" href="nm-initrd-generator.html" title="nm-initrd-generator">
<link rel="next" href="nm-openvswitch.html" title="nm-openvswitch">
<meta name="generator" content="GTK-Doc V1.33.1 (XML mode)">
<link rel="stylesheet" href="style.css" type="text/css">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table class="navigation" id="top" width="100%" summary="Navigation header" cellpadding="2" cellspacing="5"><tr valign="middle">
<td width="100%" align="left" class="shortcuts"></td>
<td><a accesskey="h" href="index.html"><img src="home.png" width="16" height="16" border="0" alt="Home"></a></td>
<td><a accesskey="u" href="manpages.html"><img src="up.png" width="16" height="16" border="0" alt="Up"></a></td>
<td><a accesskey="p" href="nm-initrd-generator.html"><img src="left.png" width="16" height="16" border="0" alt="Prev"></a></td>
<td><a accesskey="n" href="nm-openvswitch.html"><img src="right.png" width="16" height="16" border="0" alt="Next"></a></td>
</tr></table>
<div class="refentry">
<a name="nm-cloud-setup"></a><div class="titlepage"></div>
<div class="refnamediv"><table width="100%"><tr>
<td valign="top">
<h2><span class="refentrytitle">nm-cloud-setup</span></h2>
<p>nm-cloud-setup — Overview of Automatic Network Configuration in Cloud</p>
</td>
<td class="gallery_image" valign="top" align="right"></td>
</tr></table></div>
<div class="refsect1">
<a name="id-1.2.15.4"></a><h2>Overview</h2>
<p>When running a virtual machine in a public cloud environment, it is
    desirable to automatically configure the network of that VM.
    In simple setups, the VM only has one network interface and the public
    cloud supports automatic configuration via DHCP, DHCP6 or IPv6 autoconf.
    However, the virtual machine might have multiple network
    interfaces, or multiple IP addresses and IP subnets
    on one interface which cannot be configured via DHCP. Also, the administrator
    may reconfigure the network while the machine is running. NetworkManager's
    nm-cloud-setup is a tool
    that automatically picks up such configuration in cloud environments and updates the network
    configuration of the host.</p>
<p>Multiple cloud providers are supported. See <a class="xref" href="nm-cloud-setup.html#providers" title="Supported Cloud Providers">the section called “Supported Cloud Providers”</a>.</p>
</div>
<div class="refsect1">
<a name="id-1.2.15.5"></a><h2>Use</h2>
<p>The goal of nm-cloud-setup is to be configuration-less and work automatically.
    All you need is to opt-in to the desired cloud providers (see <a class="xref" href="nm-cloud-setup.html#env" title="Environment Variables">the section called “Environment Variables”</a>)
    and run <span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span>.</p>
<p>Usually this is done by enabling the nm-cloud-setup.service systemd service
    and let it run periodically. For that there is both a nm-cloud-setup.timer systemd timer
    and a NetworkManager dispatcher script.</p>
</div>
<div class="refsect1">
<a name="id-1.2.15.6"></a><h2>Details</h2>
<p>
    nm-cloud-setup configures the network by fetching the configuration from
    the well-known meta data server of the cloud provider. That means, it already
    needs the network configured to the point where it can reach the meta data
    server. Commonly that means, that a simple connection profile is activated
    that possibly uses DHCP to get the primary IP address. NetworkManager will
    create such a profile for ethernet devices automatically if it is not configured
    otherwise via <code class="literal">"no-auto-default"</code> setting in NetworkManager.conf.
    One possible alternative may be to create such an initial profile with
    <span class="command"><strong>nmcli device connect "$DEVICE"</strong></span> or
    <span class="command"><strong>nmcli connection add type ethernet ...</strong></span>.
    </p>
<p>
    By setting the user-data <code class="literal">org.freedesktop.nm-cloud-setup.skip=yes</code>
    on the profile, nm-cloud-setup will skip the device.
    </p>
<p>nm-cloud-setup modifies the run time configuration akin to <span class="command"><strong>nmcli device modify</strong></span>.
    With this approach, the configuration is not persisted
    and only preserved until the device disconnects.</p>
<div class="refsect2">
<a name="id-1.2.15.6.5"></a><h3>/usr/libexec/nm-cloud-setup</h3>
<p>The binary <span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span> does most of the
      work. It supports no command line arguments but can be configured via environment
      variables.
      See <a class="xref" href="nm-cloud-setup.html#env" title="Environment Variables">the section called “Environment Variables”</a> for the supported environment variables.</p>
<p>By default, all cloud providers are disabled unless you opt-in by enabling one
      or several providers. If cloud providers are enabled, the program
      tries to fetch the host's configuration from a meta data server of the cloud via HTTP.
      If configuration could be not fetched, no cloud provider are detected and the
      program quits.
      If host configuration is obtained, the corresponding cloud provider is
      successfully detected. Then the network of the host will be configured.</p>
<p>It is intended to re-run nm-cloud-setup every time when the configuration
      (maybe) changes. The tool is idempotent, so it should be OK to also run it
      more often than necessary. You could run <span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span>
      directly. However it may be preferable to restart the nm-cloud-setup systemd
      service instead or use the timer or dispatcher script to run it periodically (see below).</p>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.6.6"></a><h3>nm-cloud-setup.service systemd unit</h3>
<p>Usually <span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span> is not run directly,
      but only by <span class="command"><strong>systemctl restart nm-cloud-setup.service</strong></span>. This
      ensures that the tool only runs once at any time. It also allows to integrate
      with the nm-cloud-setup systemd timer,
      and to enable/disable the service via systemd.</p>
<p>As you need to set environment variable to configure nm-cloud-setup binary,
      you can do so via systemd override files. Try <span class="command"><strong>systemctl edit nm-cloud-setup.service</strong></span>.</p>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.6.7"></a><h3>nm-cloud-setup.timer systemd timer</h3>
<p><span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span> is intended to run
      whenever an update is necessary. For example, during boot when when
      changing the network configuration of the virtual machine via the cloud
      provider.</p>
<p>One way to do this, is by enabling the nm-cloud-setup.timer systemd timer
      with <span class="command"><strong>systemctl enable --now nm-cloud-setup.timer</strong></span>.</p>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.6.8"></a><h3>/usr/lib/NetworkManager/dispatcher.d/90-nm-cloud-setup.sh</h3>
<p>There is also a NetworkManager dispatcher script that will
      run for example when an interface is activated by NetworkManager.
      Together with the nm-cloud-setup.timer systemd timer this
      script is to automatically pick up changes to the network.</p>
<p>The dispatcher script will do nothing, unless the systemd service is
      enabled. To use the dispatcher script you should therefor run
      <span class="command"><strong>systemctl enable nm-cloud-setup.service</strong></span> once.</p>
</div>
</div>
<div class="refsect1">
<a name="env"></a><h2>Environment Variables</h2>
<p>The following environment variables are used to configure <span class="command"><strong>/usr/libexec/nm-cloud-setup</strong></span>.
    You may want to configure them with a drop-in for the systemd service.
    For example by calling <span class="command"><strong>systemctl edit nm-cloud-setup.service</strong></span>
    and configuring <code class="literal">[Service] Environment=</code>, as described in
    <span class="citerefentry"><span class="refentrytitle">systemd.exec</span>(5)</span>
    manual.</p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><p><code class="literal">NM_CLOUD_SETUP_LOG</code>: control the logging verbosity. Set it
          to one of <code class="literal">TRACE</code>, <code class="literal">DEBUG</code>, <code class="literal">INFO</code>,
          <code class="literal">WARN</code>, <code class="literal">ERR</code> or <code class="literal">OFF</code>. The program
          will print message on stdout and the default level is <code class="literal">WARN</code>. When
          run as systemd service, the log will be collected by journald can can be seen with
          <span class="command"><strong>journalctl</strong></span>.</p></li>
<li class="listitem"><p><code class="literal">NM_CLOUD_SETUP_AZURE</code>: boolean, whether Microsoft Azure support is enabled. Defaults
          to <code class="literal">no</code>.</p></li>
<li class="listitem"><p><code class="literal">NM_CLOUD_SETUP_EC2</code>: boolean, whether Amazon EC2 (AWS) support is enabled. Defaults
          to <code class="literal">no</code>.</p></li>
<li class="listitem"><p><code class="literal">NM_CLOUD_SETUP_GCP</code>: boolean, whether Google GCP support is enabled. Defaults
          to <code class="literal">no</code>.</p></li>
<li class="listitem"><p><code class="literal">NM_CLOUD_SETUP_ALIYUN</code>: boolean, whether Alibaba Cloud (Aliyun) support is enabled. Defaults
          to <code class="literal">no</code>.</p></li>
</ul></div>
</div>
<div class="refsect1">
<a name="debugging"></a><h2>Debugging</h2>
<p>Enable debug logging by setting <code class="literal">NM_CLOUD_SETUP_LOG</code> environment variable to <code class="literal">TRACE</code>.</p>
<p>In the common case where nm-cloud-setup is running as systemd service, this can be done via <span class="command"><strong>systemctl edit nm-cloud-setup.service</strong></span>
    and add <code class="literal">Environment=NM_CLOUD_SETUP_LOG=TRACE</code> to the <code class="literal">[Service]</code> section. Afterwards, the log can
    be found in syslog via <code class="literal">journalctl</code>. You may also want to enable debug logging in NetworkManager as descibed
    in the DEBUGGING section in <a class="link" href="NetworkManager.html" title="NetworkManager"><span class="citerefentry"><span class="refentrytitle">NetworkManager</span>(5)</span></a>
    manual. When sharing logs, it's best to share complete logs and not preemptively filter for NetworkManager or nm-cloud-setup logs.</p>
</div>
<div class="refsect1">
<a name="deploy"></a><h2>Example Setup for Configuring and Predeploying nm-cloud-setup</h2>
<p>As detailed before, nm-cloud-setup needs to be explicitly enabled. As it
      runs as a systemd service and timer, that basically means to enable and configure
      those. This can be done by dropping the correct files and symlinks to disk.
    </p>
<p>
      The following example enables nm-cloud-setup for Amazon EC2 cloud:
      </p>
<pre class="programlisting">
dnf install -y NetworkManager-cloud-setup

mkdir -p /etc/systemd/system/nm-cloud-setup.service.d
cat &gt; /etc/systemd/system/nm-cloud-setup.service.d/10-enable-ec2.conf &lt;&lt; EOF
[Service]
Environment=NM_CLOUD_SETUP_EC2=yes
EOF

# systemctl enable nm-cloud-setup.service
mkdir -p /etc/systemd/system/NetworkManager.service.wants/
ln -s /usr/lib/systemd/system/nm-cloud-setup.service /etc/systemd/system/NetworkManager.service.wants/nm-cloud-setup.service

# systemctl enable nm-cloud-setup.timer
mkdir -p /etc/systemd/system/timers.target.wants/
ln -s /etc/systemd/system/timers.target.wants/nm-cloud-setup.timer /usr/lib/systemd/system/nm-cloud-setup.timer

# systemctl daemon-reload
      </pre>
<p>
    </p>
</div>
<div class="refsect1">
<a name="providers"></a><h2>Supported Cloud Providers</h2>
<div class="refsect2">
<a name="id-1.2.15.10.2"></a><h3>Amazon EC2 (AWS)</h3>
<p>For AWS, the tools tries to fetch configuration from <code class="literal">http://169.254.169.254/</code>. Currently, it only
      configures IPv4 and does nothing about IPv6. It will do the following.</p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><p>First fetch <code class="literal">http://169.254.169.254/latest/meta-data/</code> to determine whether the
          expected API is present. This determines whether EC2 environment is detected and whether to proceed
          to configure the host using EC2 meta data.</p></li>
<li class="listitem"><p>Fetch <code class="literal">http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/</code> to get the list
          of available interface. Interfaces are identified by their MAC address.</p></li>
<li class="listitem"><p>Then for each interface fetch <code class="literal">http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/subnet-ipv4-cidr-block</code>
          and <code class="literal">http://169.254.169.254/2018-09-24/meta-data/network/interfaces/macs/$MAC/local-ipv4s</code>.
          Thereby we get a list of local IPv4 addresses and one CIDR subnet block.</p></li>
<li class="listitem">
<p>Then nm-cloud-setup iterates over all interfaces for which it could fetch IP configuration.
          If no ethernet device for the respective MAC address is found, it is skipped.
          Also, if the device is currently not activated in NetworkManager or if the currently
          activated profile has a user-data <code class="literal">org.freedesktop.nm-cloud-setup.skip=yes</code>,
          it is skipped.</p>
<p>If only one interface and one address is configured, then the tool does nothing
          and leaves the automatic configuration that was obtained via DHCP.</p>
<p>Otherwise, the tool will change the runtime configuration of the device.
            </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: circle; ">
<li class="listitem"><p>Add static IPv4 addresses for all the configured addresses from <code class="literal">local-ipv4s</code> with
                prefix length according to <code class="literal">subnet-ipv4-cidr-block</code>. For example,
                we might have here 2 IP addresses like <code class="literal">"172.16.5.3/24,172.16.5.4/24"</code>.</p></li>
<li class="listitem">
<p>Choose a route table 30400 + the index of the interface and
                  add a default route <code class="literal">0.0.0.0/0</code>. The gateway
                  is the first IP address in the CIDR subnet block. For
                  example, we might get a route <code class="literal">"0.0.0.0/0 172.16.5.1 10 table=30400"</code>.</p>
<p>Also choose a route table 30200 + the interface index. This
                  contains a direct routes to the subnets of this interface.</p>
</li>
<li class="listitem"><p>Finally, add a policy routing rule for each address. For example
                  <code class="literal">"priority 30200 from 172.16.5.3/32 table 30200, priority 30200 from 172.16.5.4/32 table 30200"</code>.
                  and
                  <code class="literal">"priority 30400 from 172.16.5.3/32 table 30400, priority 30400 from 172.16.5.4/32 table 30400"</code>
                  The 30200+ rules select the table to reach the subnet directly, while the 30400+ rules use the
                  default route. Also add a rule
                  <code class="literal">"priority 30350 table main suppress_prefixlength 0"</code>. This has a priority between
                  the two previous rules and causes a lookup of routes in the main table while ignoring the default
                  route. The purpose of this is so that other specific routes in the main table are honored over
                  the default route in table 30400+.</p></li>
</ul></div>
<p>
            With above example, this roughly corresponds for interface <code class="literal">eth0</code> to
            <span class="command"><strong>nmcli device modify "eth0" ipv4.addresses "172.16.5.3/24,172.16.5.4/24" ipv4.routes "172.16.5.0/24 0.0.0.0 10 table=30200, 0.0.0.0/0 172.16.5.1 10 table=30400" ipv4.routing-rules "priority 30200 from 172.16.5.3/32 table 30200, priority 30200 from 172.16.5.4/32 table 30200, priority 20350 table main suppress_prefixlength 0, priority 30400 from 172.16.5.3/32 table 30400, priority 30400 from 172.16.5.4/32 table 30400"</strong></span>.
            Note that this replaces the previous addresses, routes and rules with the new information.
            But also note that this only changes the run time configuration of the device. The
            connection profile on disk is not affected.
          </p>
</li>
</ul></div>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.10.3"></a><h3>Google Cloud Platform (GCP)</h3>
<p>
        For GCP, the meta data is fetched from URIs starting with <code class="literal">http://metadata.google.internal/computeMetadata/v1/</code> with a
        HTTP header <code class="literal">"Metadata-Flavor: Google"</code>.
        Currently, the tool only configures IPv4 and does nothing about IPv6. It will do the following.
      </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><p>First fetch <code class="literal">http://metadata.google.internal/computeMetadata/v1/instance/id</code> to detect whether the tool
          runs on Google Cloud Platform. Only if the platform is detected, it will continue fetching the configuration.</p></li>
<li class="listitem"><p>Fetch <code class="literal">http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/</code> to get the list
          of available interface indexes. These indexes can be used for further lookups.</p></li>
<li class="listitem"><p>Then, for each interface fetch <code class="literal">http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/mac</code> to get
          the corresponding MAC address of the found interfaces. The MAC address is used to identify the device later on.</p></li>
<li class="listitem"><p>Then, for each interface with a MAC address fetch <code class="literal">http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/</code>
          and then all the found IP addresses at <code class="literal">http://metadata.google.internal/computeMetadata/v1/instance/network-interfaces/$IFACE_INDEX/forwarded-ips/$FIPS_INDEX</code>.
          </p></li>
<li class="listitem">
<p>At this point, we have a list of all interfaces (by MAC address) and their configured IPv4 addresses.</p>
<p>For each device, we lookup the currently applied connection in NetworkManager. That implies, that the device is currently activated
          in NetworkManager. If no such device was in NetworkManager, or if the profile has user-data <code class="literal">org.freedesktop.nm-cloud-setup.skip=yes</code>,
          we skip the device. Now for each found IP address we add a static route "$FIPS_ADDR/32 0.0.0.0 100 type=local" and reapply the change.</p>
<p>The effect is not unlike calling <span class="command"><strong>nmcli device modify "$DEVICE" ipv4.routes "$FIPS_ADDR/32 0.0.0.0 100 type=local [,...]"</strong></span> for all relevant
          devices and all found addresses.</p>
</li>
</ul></div>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.10.4"></a><h3>Microsoft Azure</h3>
<p>
        For Azure, the meta data is fetched from URIs starting with <code class="literal">http://169.254.169.254/metadata/instance</code> with a
        URL parameter <code class="literal">"?format=text&amp;api-version=2017-04-02"</code> and a HTTP header <code class="literal">"Metadata:true"</code>.
        Currently, the tool only configures IPv4 and does nothing about IPv6. It will do the following.
      </p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><p>First fetch <code class="literal">http://169.254.169.254/metadata/instance?format=text&amp;api-version=2017-04-02</code> to detect whether the tool
          runs on Azure Cloud. Only if the platform is detected, it will continue fetching the configuration.</p></li>
<li class="listitem"><p>Fetch <code class="literal">http://169.254.169.254/metadata/instance/network/interface/?format=text&amp;api-version=2017-04-02</code> to get the list
          of available interface indexes. These indexes can be used for further lookups.</p></li>
<li class="listitem"><p>Then, for each interface fetch <code class="literal">http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/macAddress?format=text&amp;api-version=2017-04-02</code>
          to get the corresponding MAC address of the found interfaces. The MAC address is used to identify the device later on.</p></li>
<li class="listitem"><p>Then, for each interface with a MAC address fetch <code class="literal">http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/?format=text&amp;api-version=2017-04-02</code>
          to get the list of (indexes of) IP addresses on that interface.
          </p></li>
<li class="listitem"><p>Then, for each IP address index fetch the address at
          <code class="literal">http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/ipAddress/$ADDR_INDEX/privateIpAddress?format=text&amp;api-version=2017-04-02</code>.
          Also fetch the size of the subnet and prefix for the interface from
          <code class="literal">http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/address/?format=text&amp;api-version=2017-04-02</code>.
          and
          <code class="literal">http://169.254.169.254/metadata/instance/network/interface/$IFACE_INDEX/ipv4/subnet/0/prefix/?format=text&amp;api-version=2017-04-02</code>.
          </p></li>
<li class="listitem">
<p>At this point, we have a list of all interfaces (by MAC address) and their configured IPv4 addresses.</p>
<p>Then the tool configures the system like doing for AWS environment. That is, using source based policy routing
          with the tables/rules 30200/30400.</p>
</li>
</ul></div>
</div>
<hr>
<div class="refsect2">
<a name="id-1.2.15.10.5"></a><h3>Alibaba Cloud (Aliyun)</h3>
<p>For Aliyun, the tools tries to fetch configuration from <code class="literal">http://100.100.100.200/</code>. Currently, it only
      configures IPv4 and does nothing about IPv6. It will do the following.</p>
<div class="itemizedlist"><ul class="itemizedlist" style="list-style-type: disc; ">
<li class="listitem"><p>First fetch <code class="literal">http://100.100.100.200/2016-01-01/meta-data/</code> to determine whether the
          expected API is present. This determines whether Aliyun environment is detected and whether to proceed
          to configure the host using Aliyun meta data.</p></li>
<li class="listitem"><p>Fetch <code class="literal">http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/</code> to get the list
          of available interface. Interfaces are identified by their MAC address.</p></li>
<li class="listitem"><p>Then for each interface fetch <code class="literal">http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/vpc-cidr-block</code>,
          <code class="literal">http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/private-ipv4s</code>,
          <code class="literal">http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/netmask</code> and
          <code class="literal">http://100.100.100.200/2016-01-01/meta-data/network/interfaces/macs/$MAC/gateway</code>.
          Thereby we get a list of private IPv4 addresses, one CIDR subnet block and private IPv4 addresses prefix.</p></li>
<li class="listitem">
<p>Then nm-cloud-setup iterates over all interfaces for which it could fetch IP configuration.
          If no ethernet device for the respective MAC address is found, it is skipped.
          Also, if the device is currently not activated in NetworkManager or if the currently
          activated profile has a user-data <code class="literal">org.freedesktop.nm-cloud-setup.skip=yes</code>,
          it is skipped. Also, there is only one interface and one IP address, the tool does nothing.</p>
<p>Then the tool configures the system like doing for AWS environment. That is, using source based policy routing
          with the tables/rules 30200/30400. One difference to AWS is that the gateway is also fetched via metadata instead
          of using the first IP address in the subnet.</p>
</li>
</ul></div>
</div>
</div>
<div class="refsect1">
<a name="id-1.2.15.11"></a><h2>See Also</h2>
<p>
      <a class="link" href="NetworkManager.html" title="NetworkManager"><span class="citerefentry"><span class="refentrytitle">NetworkManager</span>(8)</span></a>
      <a class="link" href="nmcli.html" title="nmcli"><span class="citerefentry"><span class="refentrytitle">nmcli</span>(1)</span></a>
    </p>
</div>
</div>
<div class="footer">
<hr>Generated by GTK-Doc V1.33.1</div>
</body>
</html>